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ABSTRACT 

Details are given of a seven-diget coding system used 
to analyze instructional materials in elementary school mathematics. 
Four variables are analyzed: the first four digits of the code 
identify a mathematical concept or skill; the fifth digit identifies 
the highest cognitive level expected of the student in completing the 
task; the sixth digit identifies the stage of mastery at which the 
pupil is expected to perform; and the seventh digit of the code gives 
the mode of pupil response. Lists of numbers keyed to specified 
cognitive verbs^ stages of mastery^ and modes of response are 
included in this document. An experiment in v;hich pre-service 
mathematics education majors used this system to code instructional 
materials is briefly described, (DT) 
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Many systems have been devised for analysis of communications for their 
manifest content. Typically such analysis is performed with respect to one or 
more variables in which different units of analysis miqht be exoected to exhib- 
it differences. For example, during World War II, certain domestic and foreign 
publications were analyzed for their oropaganda content. When material is 
analyzed, a code is often used to indicate the presence of a given oronprtv. 
Coding sources define the codes and are used in the analysis process. The cod- 
ing sources, together with instructions for their use, are said to constitute 
the coding system. 

The sequence of instructional activities to be described is based unon a 
coding system whose ourpose is the analysis of instructional materials in elemen- 
tary school mathematics. The coding system is based upon a seven-digit code and 
analyzes instructional materials for four variables, each of which is described 
below. A description of the sequence, experimental evidence for- its effective- 
ness, and implications for further study follow. 

The coding system is based on a seven-digit code and designed for elementarv 
school mathematics curricula. The principles of the coding system can, ho\/evor, 
easily be applied in other areas of instruction. 

The first four digits of the seven-digit code identify a mathematical con- 
cept or skill called content as defined by the Mathematics Content Authority 
List: K-6 (PRIMES, Pennsylvania Department of Education) of which Fiaure I is a 
sample page. 

The fifth digit identifies the highest cognitive level expected of the 
pupil in completing the task as defined by the Cognitive Verb List (Figure II). 
Any level two or higher verb requires the use of some lower level verb as a com- 
ponent behavior. For example, compare must be used as a tool in both illustrate 
and categorize. Thus there is evidence for the contention that the Coonitive 
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0.160 (contd) .... (3) The associative property is some- 
tines called the grouping property. 

Ex. The operation of addition of 
whole numbers is associative. 

5+2+1 = (5+2)+l = 5+(2+l) 

7 + 1 = 5 + 3 
8 = 8 

a + b + c = (a+b) + c = a + (b+c) 
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^^^^ g) Zero, the identity element in ad- 

dition 

The identity element in addition 
is the number which v;hen added to 
any number leaves that number un- 
changed • 

Ex. 3+0=0+3=3 
n + 0 = 0 + n= n 
0 is the identity element 
for addition. 

h) Role of one in addition 

When 1 is added to a whole number 
the sum is the next greater whole 
number. 

Ex. 23 + 1 = 24 102 + 1 » 103 

0182 2) Computation 

0184 a) Two addends 

0150 (1) Elementary facts of addition 

An elementary (basic) fact of 
addition has two whole number 
addends, each less than ten. 

Ex. 6 + 7 « 13 



FIGURE I 



SAMPLE PAGE FROM THE MATHEMATICS CONTENT 
AUTHORITY LIST: K-6 
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February 18, 1970 



LEVEL 9 9e 
LEVEL 8 8p 
8t 

LEVEL 7 7h 
7s 
7g 

LEVEL 6 6j 
6e 
6a 

LEVEL 5 5a 
5r 
5c 
5u 
5d 

LEVEL 4 4c 
4i 

LEVEL 3 3c 
3s 

LEVEL 2 2i 
LEVEL 1 la 
lo 
Ip 

LEVEL 0 On 



COGNITIVE VERB LIST 
EVALUATE 

PROVE 8s 
TEST (experiment) 8d 
HYPOTHESIZE (theorize) 
SYNTHESIZE (organize, structure) 

7d 



SOLVE 
DESIGN 



DEDUCE 



5y 



SYMBOLIZE 



GENERALIZE (induce) 
JUSTIFY (support) 
EXPLAIN (interpret) 
ANALYZE 
APPLY 
RELATE 

CONVERT (translate) 
SUMMARIZE (abstract) 
DESCRIBE 

CATEGORIZE (classify, group) 

ILLUSTRATE (exemplify) 

COMPARE 

SUBSTITUTE 

Iterate 

RECALL 

RECOGNIZE 

repeat; copy (imitate, reproduce) 

NO TASK Oo OBSERVE 



Or 



READ 



ERIC 



FIGURE II 
COGNITIVE VERB LIST 



STAGES OV MASTERY LIST 



0 NOT APPLICABLE 

1 READINESS 



DEVELOPMENT 



PRACTICE 



DEMONSTRATION 



MAINTENANCE 



development, practice, demon- 
stration, or maintenance of pre- 
requisite skills to be used in the 
development of another topic, 
(Content coded is the pre-requisite 
skill.) 

generating or evolving concepts. 
Student demonstrates the skills 
necessary to contribute to the 
development • 

performance of behaviors (not for 
the first time) prior to demon- 
stration of mastery or prior to 
demons rat ion of over-learning. 

performance of behaviors at or 
beyond established criteria, 
(test) 

practice after a lapse of time to 
enable students to retain be- 
haviors at established criteria, 
(review) 



FIGURE III 
STAGES OF MASTERY LIST 
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Verb List is hierarchical v/ith respect to complexity of expected student behav- 
ior. 

The sixth digit identifies the stage of mastery at which the nunil is 
expected to perform as defined on the Stages of Mastery List (Figure III). If 
content and cognitive level are held constant, then stage of master/ codes one 
through five represent a chronological sequence from readiness through devolon- 
ment to practice to demonstration to maintenance. 

The seventh digit of the code gives the mode of puDil response as defined 
by the Modes of Response List (Figure IV), 

Because cognitive levels and stages of mastery can be ordered, the seouenco 
of activities by v;hich members of the experimental group v;ere taught to code 
were ordered by these variables. 

The sequence consists of 50 activities. Each activity is represented hv on 
ordered pair (m, n) of whole numbers where m stands for a coanitive level and n 
a stage of mastery. For example, the activity represented by (1, 3) would be 
some activity with a level 1 cognitive verb and level 3, or practice, stage of 
mastery. The order of activities is given by the sequence of ordered pairs 
(Figure V). The choice of a particular cognitive verb at any level v/as dictated 
by oractical considerations. 

Every ordered pair except the first and last lias on some diagonal of the 
Matrix of Coder Training Activities (Figure VI). The ordered pairs on anv diao- 
onal are grouped together by brackets in the ordered pair sequence (Figure V). 

Learning experiences have been devised for each ordered pair in the sonuence. 
For example, the activity associated with the ordered oair (1, 3) renuires stu- 
dents to practice recognizing cognitive verbs that occur in instructional mate- 
rials to v;hich they are referred. They are asked to respond by v/riting on their 
answer sheets those underlined v/ords in the instructional material that thev 
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MODES OF RESPONSE LIST 

0 NO OVERT MODE OF RESPONSE 

1 CHECK, RING (encircle), UNDERLINE, CROSS OUT, I-IARK, 
CONNECT (with a line segment), GROUP 

2 PUT, TAKE, AFFIX, PLACE 

3 DRAW, GRAPH, TRACE, DIAGRAM 

4 COLOR, SHADE 

5 WRITE 

6 POINT AT 

7 SAY 

8 MANIPULATE, CUT, FOLD 

9 DR/.MATIZE, SIMULATE 



FIGURE IV 



MODES OF RESPONSE LIST 
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(read, readiness) 3 


(7,1), 


(generalize, readiness) 


1 




(recognize, readiness) 1 




(6,2), 


(explain, development) 


[(0.2), 


(read, development) J 




(5,3), 


(apply, practice) 




(2,1), 


(iterate, readiness) 




(4,4), 


(categorize, demonstration) 


(1,2), 


(recognize, development) 




(.3,5), 


(compare, maintenance) 


_ 


(0,3), 


(read, practice) 




(8,1), 


(solve, readiness) 




(3,1) , 


(compare, readiness) 




(7,2), 


(deduce, development) 




(2,2) , 


(Iterate, development) 




(6,3), 


(analyze, practice) 




(j-,3) , 


(recognize, practice) 




(5,4), 


(apply, demonstration) 




(0,^) , 


(read, demonstration) 




L4.5), 


(categorize, maintenance) 




/J "IN 
(^,1), 


(categorize, readiness) 




^9,1), 


(evaluate, readiness) 




(3,2), 


( compa re , development) 




(8,2), 


(design, development) 




(2,3) , 


(Iterate, practice) 




(7,3), 


(synthesize, practice) 






(recall , demonstration) 




(6,4), 


(analyze, demonstration) 






(read, maintenance) 




(5.5), 


(apply, maintenance) 




(5,1), 


(apply, readiness) 




r9,2). 


(evaluate, development) 


- 


(^,2), 


(categorize, development] 




(8,3) , 


(design, practice) 




w, •>) , 


(compare , practice) 




(7,4), 


(synthesize, demonstration) 


(2,4), 


(Iterate, demonstration) 




L6.5), 


(analyze, maintenance) 




(1,5), 


(recall, maintenance) 




(9,3), 


(evaluate, practice) 




(6,1), 


(justify, readiness) 




(8,4), 


(design, demonstration) 




(5,2), 


(apply, development) 




17,5), 


(hypothesize, maintenance) 


(4,3), 


(categorize, practice 




(9,4), 


(evaluate, demonstration) 




(3,4), 


(compare, demonstration) 




(8,5), 


(desl{»n, maintenance) 




.(2,5), 


(iterate, maintenance) 


[9,5), 


(evaluate, maintenance) 


] 



FIGURE V 

SEQUENCE OF CODER TRAINING ACTIVITIES 
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recognize as cognitive verbs. 

Self-instructional materials v;ere written for the first tv/enty-einht of 
the fifty ordered pairs above and formed the core of the exnerimental treat- 
ment. An additional two exercises not in the sequence v;ore included for 
enrichment* 

As stated above, all behaviors represented by level tv/o and higher coa- 
nitive verbs require the use of lower level behaviors as tools. 

Therefore, when a cognitive level is introduced for the first time, stu- 
dents will already have been exposed to each lower cognitive level at succes- 
sively higher stages of mastery. Thus the stage of mastery order is preserved. 
It is judged to be too much to ask of students to increase both cognitive level 
and stage of mastery at once. It v/as also considered desirable to nresent tho 
student v/ith experiences at a variety of cognitive levels rather than to hold 
cognitive level constant an^i vary stage of mastery. This soaced activitv/ vn'th 
each cognitive level is consistent v/ith the concent of a spiral curriculum and 
reduces the probability of students becoming bored v/ith the sequence of activ- 
ities to v/hich they are exposed. This is also consistent v/ith Bruner*s conten- 
tion that "the development of intellectual functioning. . .is shaoed by a series 
of technological advances in the use of mind."^ 

The experiment by which the effectiveness of the instructional materials 
based on this design v/as tested used a nre-test, post-test, control nroun design 
All but one of the subjects v/ere college students preparina to teach mathematics 
The differences betv/een consistency gains in coding the exnerimental ^jrouo from 
ore-test to nost-test exceeded those of the control group by an amount sianifi- 
cant at the .025 level or better for every coding source. 

See Bruner, Jerome S. "The Course of Cognitive Growth," America n 
Psychologist , 19, January, 1964, n.l 



Evidence has been referred to above to suonort the hierarchical nature of 
thC: Coqnitive Verb List and the sequential nature of Staoes of Masterv List, 
learning activities were ordered according to these supported assumptions. li 
has not been demonstrated that the sequence of ordered pairs thus qenerated 
represents a sequence of activities that is of increasing challenqe to tho 
student. 

This could be investigated by asking randomly selected individuals to 
complete a sequence of tasks each of which represents an ordered oair in the 
matrix. If the ordering given by the matrix does represent a hierarchy, then 
significantly fewer individuals shouTd successfully complete higher level 
tasks than comolete lower level tasks. 

It may be that students who have undergone the experimental treatment 
will become better teachers than they otherv/ise would have been because of 
their increased sensitivity to the variables with respect to which they coded 
instructional material. This question could be investigated by analyzino the 
teaching of members of the experimental group for the presence of decisions 
about each of the four variables and comparing the results with analyses per- 
formed on the teaching behavior of other teachers. 

The experiment descrived above demonstrates that the desian by which 
instructional activities were sequenced was more effective than testing alone. 
A next step would be to compare the effectiveness of instructional activities 
based on this design with instructional activities seouenced by content onl>/. 



